1. Field of the Invention
This invention relates to a computing system for performing first and second order correlations between the inputs to and outputs from a linear or non-linear system, whereby that system may be characterized, and more particularly to improvements therein.
Techniques of linear system theory have been used extensively in the study of non-linear systems. There are two principle reasons for this: the simplicity and completeness of linear theory; and the difficulty of linking non-linear analysis with functional entities. Often such systems have been studied with linear techniques using small signal approximations or by making certain linearizing assumptions about the system's behavior. Certain specialized methods for non-linear systems' analysis exist, however all these methods have serious limitations and are applicable to rather narrow classes of non-linear systems.
Starting with Norbert Wiener's original work (Non-Linear Problems in Random Theory, Wylle, New York, 1958), a general theory of non-linear system analysis and synthesis has been in development. This theory assumes only time-invariance and finite memory, therefore it is applicable to many physical and living systems. Wiener proposed that a non-linear system could be identified by its response to a Gaussian white-noise stimulus, and with such an input, the system is tested effectively with all possible inputs, (in practice with a great variety of inputs, depending upon the length of the experiment). Wiener's original formulation wasn't practical for experimental applications (Marmarelis, 1972), but Lee and Schetzen, in a work entitled "Measurement of Kernels of a Non-Linear System by Cross Correlation," Quarterly Progress Report No. 60, Research Laboratory for Electronics, Massachusetts Institute of Technology, 1965, proposed a simpler formulation in terms of cross-correlation calculations. This modification of the Wiener technique provides a feasible approach to analyzing time-invariant, finite memory systems.
Given a system with Gaussian, white-noise input x(t) and output y(t) we have: EQU y(t)=F[x(t)]
Wiener showed that the functional F can be rewritten as the following series: ##EQU1## where P is the power spectral density of the "quasi-random" input and by definition is a constant.
The set of kernels (h.sub.0, h.sub.1, h.sub.2, . . . ) completely characterizes the system. Each kernel h.sub.n is a symmetric function of its arguments. The kernels describe quantitatively the non-linear cross talk between different portions of the past of the input as it affects the system response at the present, i.e., how much the response to n different pulses deviates from the superimposed responses to single pulses.
The above-indicated formula y(t)=F[ x(t)] indicates a method which is particularly well suited for the study of biological systems. In operation, a stimulus x, is applied to the system, and the output y, is noted. The Wiener formulation demands that the stimulus is a Gaussian random sequence which by its nature contains nearly all possible stimuli, thus the resulting characterization contains information about the system's response to nearly all stimuli. It can be shown that by multiplying ternary versions of x with values of y the h.sub.0, h.sub.1 and h.sub.2 Wiener kernels can be approximated quite closely.
With the growing interest in the study of non-linear systems, it becomes important to be able to compute these kernels. The determinations for the kernels h.sub.0 and h.sub.1 have not been found to be difficult. The h.sub.0 kernel is normally equal to the average of the response y. The h.sub.1 kernel is determined by correlating the quasi-random input x with the output y. The definition of h.sub.2 involves a double correlation involving two inputs (two x's).sup.2 and one output (y). In many cases the value of the higher order kernels above the second order is so small as to be insignificant, and thus may be disregarded.
Calculating the kernels h.sub.1 and h.sub.2 usually involves an expensive and sophisticated computer system to perform the required correlations. As indicated, the h.sub.0 kernel is simply determined by the average value of the y. The h.sub.1 kernel requires a firstorder correlation which is also simple and does not require many operations. However, the algorithms required for determining the h.sub.2 kernel requires many arithmetic operations. For example, if a first order kernel requires 32 operations, then the second order kernel typically requires 1024 or more operations. This is expensive to perform on a general purpose computer and is also very time comsuming.